Efficient coding of visual information in neural computations

Laurent Perrinet

LACONEU 2019: 5th Latin-American Summer School in Computational Neuroscience, 16/1/2019

(AUTHOR) Hello, I am Laurent Perrinet from the Institute of Neurosciences of la Timone in Marseille, a joint unit from the CNRS and the AMU
(OBJECTIVE) in this talk, I will be focus in highlighting some key challenges in understanding visual perception in terams of efficient coding using modelization and neural data
please interrupt
(ACKNO) this endeavour involves different techniques and tools ... From the head on, I wish to thanks people who collaborated and in particular .. mostly funded by the ANR horizontal V1 (fregnac chavane) + ANR TRAJECTORY (o marrre bruno cessac palacios )
LONDON (Jim Bednar, Friston)
(SHOW TITLE) I am interested in the link between the neural code and the structure of the world. in particular, for vision, I am researching the relation between the functional (in terms of the inferential processes leading to behaviour) organization (anatomy and activity) of low-level visual areas (V1) and the structures of natural scenes, that is of the images that hit the retina and which are relevant to visual perception in general.

so what is vision efficient? in particular if we look around us, images are formed by a relatively low number of features, which are arranged according to prototypical structures - lines, curves, contours

Olshausen and Field (1997) Sparse coding with an overcomplete basis set: A strategy employed by V1?

a seminal idea is proposed by Olshausen: * this may be formalized as an inference problem:

edges are different sources, which are known to be sparse: by mixing these sources one forms the image (transparency hypothesis)

the sparseness is characterized by the pdf of the sources coefficients

this inference problem is an inverse problem:
while this problem may be hard to solve, this may be approached using a (conjugate) gradient descent which has a nice implementation in terms of neural networks

I wish to point here to an essential feature compared to classical feed-forward networks: you can not do the inference in one single shot in general; you need a recurrent / recursive network (see arrow) and this is precisely a possible function for one of the most numerous type of synapses: short-ranges lateral interactions

Olshausen and Field (1997) Sparse coding with an overcomplete basis set: A strategy employed by V1?

a seminal idea is proposed by Olshausen: * this may be formalized as an inference problem:

edges are different sources, which are known to be sparse: by mixing these sources one forms the image (transparency hypothesis)

the sparseness is characterized by the pdf of the sources coefficients

this inference problem is an inverse problem:
while this problem may be hard to solve, this may be approached using a (conjugate) gradient descent which has a nice implementation in terms of neural networks

Olshausen and Field (1997) Sparse coding with an overcomplete basis set: A strategy employed by V1?

a seminal idea is proposed by Olshausen: * this may be formalized as an inference problem:

edges are different sources, which are known to be sparse: by mixing these sources one forms the image (transparency hypothesis)

the sparseness is characterized by the pdf of the sources coefficients

this inference problem is an inverse problem:
while this problem may be hard to solve, this may be approached using a (conjugate) gradient descent which has a nice implementation in terms of neural networks

Bosking et al. (1997) Journal of Neuroscience

in the primary visual cortex for instance, the set of long-range lateral connections between neurons, which could act to facilitate detection of contours matching the association field, and/or inhibit detection of other contours. To fill this role, the lateral connections would need to be orientation specific and aligned along contours, * (colin) and indeed such an arrangement has been found in treeshrew's primary visual cortex (Bosking et al., J Neurosci 17:2112-27, 1997) (neural) if one looks at the primary visual area in the occipital lobe of the cortex using optical imaging as here in the treeshrew by Bosking and colleagues under the supervision of DF, one could represent the distributed, topographical representation of orientation selectivity. in (A) and (B) the orientation giving the most response at each cortical position is represented by hue using the code below from orange for horizontal to blue for verticals, and typical structures are magnified in (C): stripes (on the periphery) and pinwheels. You can understand this as a packing of a 3D feature space on the 2D surface of the cortex. (method) Tree shrew orientation preference maps were obtained using optical imaging. Additionally, 540 nm light was used to map surface blood vessels used for alignment. Biocytin was then injected into a specific site in V1 and the animal was sacrificed 16 hours later. Slices of V1 were imaged to locate the biocytin bouton and the surface blood vessels. The blood vessel information was then used to align the orientation preference maps with the bouton images giving overlaid information on the underlying connectivity from the injection site on the animal. The original experiment used a total of ten cases. (lateral) we show here one result of Bosking which overlay over a map or orientation selectivity the network of lateral connectivity originating froma group of neurons with similar orientations and position. There is a structure in this connectivity towards locality (more pronounced for site B) + connecting iso orientations even on long ranges (A). This type of structure tends to wire together those neurons that have similar orientations, indicating a prior to colinearities. (colin) ... Overall, a typical assumption that the role of lateral interactions is to enhance the activity of neurons which are collinear : it is the so-called association field formalized in Field 93, as was for instance modeled neurally in the work from P. Series or in this version for computer vision (physio) is there a match of these structures with the statistics of natural images? 2) Some authors (Kisvarday, 1997, Chavane and Monier) even say it is weak or inexistent on a the scale of the area... 1: Hunt & Goodhill have reinterpreted above data and shown that there is more diversity than that - TRANSITION : my goal here will be to tackle this problem at different levels:

Kremkow, LP, Monier, Alonso, Aertsen, Fregnac, Masson (2016) Push-pull receptive field organization and synaptic depression: Mechanisms for reliably encoding naturalistic stimuli in V1

Outline

Efficiency, vision and neurons

Sparse coding in the retina?

Sparse Hebbian Learning

Ravello, Escobar, Palacios, LP (2019) in prep

Olshausen and Field (1997) Sparse coding with an overcomplete basis set: A strategy employed by V1?

Ravello, Escobar, Palacios, LP (2019) in prep

Outline

Efficiency, vision and neurons

Sparse coding in the retina?

Sparse Hebbian Learning

(from http://cs231n.github.io/convolutional-networks/)